ID oiri :S37 

TPB»M. 

iwsriffirioH 
mORT jio 

PUB DKTJB" 



ma FRzcB 
OBBCixsms 



iBi dio 182 

Sfainge, Jttlw 'B.: ~M ~ 
Xn^Qdqetion Ibo PvydioXpgy and Leadership. 
Specifications of iteseercb Test item. . 
BTOl Acadeay, AnaiapoIXs; Md. ; fiestinglioase Leaxnino 
corp.* JMmapolis, iNd. . ^ 
Mational center fbr Bdncational Research and 
Qe?elopii«nt (DBBK/oB), Washington, D.c. 

BR-8oO««8 

Dec $9 

B0.Q600r«8-0 1525 

Ip. also BH 01.0 «1.8. .and EN 010 119 " 
ilPT80.65 iiC«f3.29 ~ 

■^^^Qi aslTogtiqgal. Aids.; jcw^icnlim Design; 
?^SfS5?^?? Pf^^^^^i- -Ii»4lvid!»liBed. cnrriculuii: 
fo^3»^»lised ^lustinle^on; . jMiyidoal psycfaology ; 
^stroctiml .Design! .Lead^^ leadership 
^^^iqing: Hanagoient Bdiocatlon: MiUtary trainings 
motinedia ins^raction: Perfovmiice criteria 
Psychology; Social Psycbpiogy; Technical Reports; 
♦JPeat Qonstraction; ♦Tests. • 



BB8TRBCT 

. yrUpg guidelines _f or ti>e deyelppMnt of 
xmwm test it^. fqjr the connlatiye ppst-tesU of the Dttited 
£tates Naval .Acadeiqr's leadership coarse developed hy the * 

^21**'**'^ learning Ccrporation are presoited in this report, 
inclining general specifications, content analysis and test 

SS'Sy^i^^in'Sv'^f'^^i***^ ^ 010 MB through EN 010 

«A7 and BIJ10 «51 thCQogh EN 010 m jc^^t^^A dociwents; mth the 

^J^ij^** '^l^*^ ^"^"^ ™ *19., and EN OiO %8*. 




i FILMED FROM BEST AVAILABLE COPY f 



Westinghouse Learning Corporation 



fOUCATlOHlAWCUAIIf 
TM.c ^?^'''CiOFiOUCATIOIi 
^M'S OOCUMCNr HAS SEEN REMO 
OUCEO EXACriv AS «CElVEO M^M 
;,,7,rj"50N OR ORGAN.eAT,ON 'q",^^ 

NECESSAPIIIV 
JJC^HESENT O'flClAL 0«ICE Of EOU 
CATfONPOSlTfON OR POLICY 



TP - 6.9 



December 9, 1969 



O LEADERSHIP MANAGE/V£NT COURSE 

IJLi ' SPECIFICATIONS OF RESEARCH TEST ITEMS 

CONTRACT NO. N00600-68->C-1525 



ABSTRACT 

This paper represents Wesflnghouse Learning Corporation's 
working guidelines for the development of research test items for 
the Cumulative Post-Tests of the USNA AAulti-Medlo Course 
Development Project. 
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SPECIFICATIONS OF RESEARCH TEST ITEMS 
Ceneral specifications 

1. Relationship of items to content ; Cumulative Post-^-Test (CPT) items 
will be prepared on the basis of content outlines and the content of 
••common" instructional m^erials (i.e., materials presented in each 
of the parallel experimental modules). 

Each CPT item must have an identifiable relationship co one or more 
general and/or specific headings of the content outlines; references 
to content outline" headings will be noted on each test item sub-, 
mission form; 

Each item will be related to the content of as many different segments 
making up the Cumulative Post-Test Unit as is feasible and desirable 
In Velatica to other specifications detailed below. The entire set of 
items should represent an equitable distribution of content among seg- 
ments of the Unit in relation to the relative ariount of content covered 
by the outlines and the materials of each segment. 
2* Number of items p er Unit; The number of items to be developed for 
each CPT Unit will equal ten (10) times the number of segments making 
up the Unit. 

3. Item difficulty; CPT items will possess high discriminative power; 
WLC will endeavor to construct items of rel;?tiv.?ly high initial diffi- 
culty and sensitivity to gain, using the general guideline that not 
more than 40% of the students should answer the item •correctly on a 
pre-test, nor less than 50% of the students should answer the item 
correctly on a Post-Test, /ctual indices of item difficulty will be 
determined in the validation testing program at the USNA> as a b.^.^is 
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for subsequent item revision. 

4. Format of items ; CPT items will be developed in a final multiple-choice 
format; the student Will be required to select the one correct or most 
appropriate answer from among four possible alternatives. 

5. Types of items; CPTs will include items which represent "aco dsltion 
of knowledge" (Type I items) and "application of knowledge" (Type II 
items) . 

Type I items will measure acquisition of specific factual information 
covered in one or more segments of a Unit. Acceptable forms of Type 

I items include definition-identification, discrimination-comparison, 
multiple discrimination, and concept identification questions. 

Type II items will measure application of concepts and principles 
covered in one or more segments of a Unit. Acceptable forms of Type 

II items include problem identification (selection of the correct or 
most appropriate example or illustration of a given concept or prin- 
ciple) and problem solving (selection of the correct or most approp- 
riate solution of a given problem). Each CPT will contain approximately 
equal nuirbers of Type I and Type II test items. 

6. Length of items; CPT items will be concise; they will contain only 
essential details. 

As a specific guideline, each CPT item will be constructed so that a 
USNA midshipman may reasonably be expected to read, understand, and 
respond to the item within one minute. 
Content Analysis and Item Constructio n . • 

1. Analysis of content; The CPT should measure acquisition and applica- 
tion of content within segments, and also the ability to integrate and 
utilize material presented in several segments of the Unit. 
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Construction of a CPT will therefore depend upon an analysis of Unit 
content. 



WLC will examine all terms, concepts, and principles contained in 
the Unit, together with their associated definitions, descriptions, 
purposes, advantages, disadvantages, and/or uses. 

Groups of terms, concepts, principles etc., which Have similar and/or 
potentially conf usable elements will be identified. Test items can 
then be constructed which measure, in a multiple-choice format, the 

i 

ability to identify, discrimin^itp between, compare, and apply in 
problem situations, elements of the Unit content. 

Other relationships among terms, concepts, principles, etc., of the 
Unit will also be examined, so that test items can be constructed 
which measure, in a multiple-choice format, the ability to generalij^e 
from, integrate, and apply material presented in the Unit. 
C. Classification of items 

1. Type I items (acquisition of knowledge) 

a. Definition-identification: Item will require selection of the 
correct definition, description, purpose, or use of a given 
term, concept, or principle; 

or: 

Item will require selection of the correct terra, concept, or 
principle which is defined or described by a given definition 
or description. 

b. Discrimination-comparison: Item will require selection of the 
correct distinction between or comparison of a given sot of 
terms, concepts, and/or principles; 

or: 

Item will require the correct matching of a set of terms, con-- 



cepts, and/or principles with a set of definitions and/or 

descriptions, as: . 

Which matching of words and statements is correct? 

1. Term 1 A. Definition 1 

2. Term 2 B. Definition 2 

3. Term 3 C. Definition 3 
Term 4 D. Definition 4 

(possible answers, one of which is correct): 

a) 1-B, 2-D, 3-B 

b) 1-D, 2-B, 3-A 

c) 1-D, 2-B, 3-C 

d) 1-C, 2-D, 3-A 

2. Type II items (application of knowledge) 

a. Generalization-problem identification: Item will require selection 
of the correct or most appropriate "real-life" application, example 
or illustration of a given concept or principle; 

or: 

Item will require selection of the correct concept I or principle 
Illustrated by a given "real-life" example or illustration. 

or: 

Item will require correct matching of a set of concepts and/or 
principles with a set of "real-life" examples and/or illustrations. 

b. Problem solving: Item will require selection of the correct or 
most appropriate solution of, resolution of, or reaction to a 
given "real-life" problem. or situation; 

or: 

Item will require correct matching of a set of concepts and/or . 
principles v-lth a set of solutions of, resolutions of, and/or ' 



reaction to a given "real-life" i-ioblan or situation (i.e., how 
vould alternative theories, methods, or approaches deal with the 
same situation or problem). 
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